Home

Appendix

Application Icon   Available Plugin Keys

Plugins created in DEVONagent Pro's Plugins & Scanners window, are saved in ~/Library/Application Support/DEVONagent/Plugins. The format of the plugin depends on whether you choose to edit the plugin in Plist (.xml) or JSON (.json) format. Optionally, you can work in an external application. XML-based plugins can be created or edited in applications like BBEdit or even Apple's Xcode. JSON-based plugins are plain text and can be created and edited in any plain text editor, for example TextEdit. Externally created plugins should be saved with the proper file extension in the same location mentioned above.

When you create a plugin in the application, certain keys will be automatically added for you. Double-click the plugin in the Plugins & Scanners window to view and edit the keys. This is a good way to learn the internal structure and workings of a DEVONagent Pro plugin. It's easy to make changes and test them quickly using this window. However, you can use in an external editor, if you choose.

Listed below are the available keys, some required and some optional. Also, if the source to be processed is not HTML but XML or JSON output, there are specific keys that need to be used. Those are listed separately.

Boolean values are true or false and string values are case-insensitive.

Keys for HTML

Required Keys:

  • Icon
    Name: [string or dict] The name of the plugin. Example: BBC News.
  • Icon
    Info: [string or dict] Information containing name, version and copyright. Example: BBC News Plugin 1.0 © 2003--2005 DEVONtechnologies.
  • Icon
    Description: [string or dict] Describes the functionality of the plugin. Example: Queries the BBC news archive.
  • Icon
    Version: [string] The version number (format major.minor). Example: 1.0.
  • Icon
    Identifier: [string] Unique identifier (format domain.name.plugin). Example: com.devon-technologies.bbcnews.plugin.
  • Icon
    EngineUrl: [string or array of strings] The Internet address(es) of the search engine. The strings _agentQuery_, _agentNumber_ and _agentOffset_ are replaced by the search term, the number of results and the offset of the current page. _agentNumber_ and _agentOffset_ are optional. _agentLanguage_ is replaced by the language used by the plugin (ISO 639 two-letter codes). Can contain the placeholders _agentUser_ and _agent_Password_ for HTTP authentication.

Example:

https://www.bbc.co.uk/cgi-bin/search/results.pl? tab=news&start=_agentOffset_& go=homepage&q=_agentQuery_.

Optional Keys:

  • Icon
    CrawlDelay: [float] Specify a delay between two requests sent to the same server, defined in seconds.
  • Icon
    DateEnd: [string or array of strings] String marking the end of the range of the HTML source code containing the date of a page.
  • Icon
    DateStart: [string or array of strings] String marking the beginning of the range of the HTML source code containing the date of a page. Note that DateEnd is optional if DateStart is specified. Useful if server provides no dates or the wrong dates.
  • Icon
    EncodingPage: [string] Default character encoding for result pages (default: Latin-1). Examples: ISO-8859-1, UTF-8, ISO-8859-2, Shift-JIS.
  • Icon
    EncodingUrl: [string] Character encoding for search term _agentQuery_ (default: Latin-1). Examples: ISO-8859-1, UTF-8, ISO-8859-2, Shift-JIS.
  • Icon
    EngineNextUrl: [string or array of strings] The Internet address(es) of the search engine(s) used for subsequent pages (if not identical to the address of the first page). The strings _agentQuery_, _agentNumber_ and _agentOffset_ are replaced by the search term, the number of results and the offset of the current page. _agentNumber_ is optional.
  • Icon
    EngineRelated: [string] Defines a URL for a 'see also' function of the search engine.
  • Icon
    EngineSite: [string or array of strings] Defines (a) (sub)domain(s) with or without a path to search. DEVONagent Pro uses an appropriate common search engine to perform the search. Example: devontechnologies.com/products/.
  • Icon
    ExcludeLinksNamed: [array of strings] Defines an array of names of links which should be skipped.
  • Icon
    FileTypes: [string] List of all supported file types: html, feed/rss, text, pdf, OpenOffice, Microsoft/Word, RTF or all. Default is all.
  • Icon
    FilterLanguage: [Boolean] Enable or disable filtering by language.
  • Icon
    FilterSimilarPages: [Boolean] Enable or disable DEVONagent Pro's Similar Pages filter (default: true).
  • Icon
    FollowLinks: [Boolean] Enable or disable DEVONagent Pro's Follow Links option (default: true).
  • Icon
    GoogleTags: [Boolean] Defines whether googleon/googleoff tags should be supported.
  • Icon
    HTTPShouldHandleCookies: [Boolean] Defines whether cookies should be handled by HTTP or not.
  • Icon
    Keyword: [string or array] One or more unique keywords used for quick searching (like "Mozilla Keywords") by entering the keyword followed by the search term in the URL field of the browser window.
  • Icon
    KeywordsUrl: [string] Defines an alternate URL used for the Mozilla-like keywords in the browser's address bar.
  • Icon
    LinksEnd: [string or array of strings] String marking the end of the range of the HTML source code containing the links to use (any position after the last link to use but before the next link). Example: -- Footer Area --.
  • Icon
    LinksMatching: [string or array] Only links of result pages matching this string or these strings are used (prefix/suffix * wildcards are supported). Examples: *news.bbc.co.uk/*.
  • Icon
    LinksNotMatching: [string or array] Only links of result pages not matching this string or these strings are used (prefix/suffix * wildcards are are supported).
  • Icon
    LinksStart: [string or array of strings] String marking the beginning of the range of the HTML source code containing the links to use (any position before the first link to use and after the last link to ignore). Note that LinksEnd is optional if LinksStart is specified. Example: search results for.
  • Icon
    LoginPost: [string] Defines the optional HTTP POST parameters. Usage like Post. Can contain the placeholders _agentUser_ and _agent_Password_.
  • Icon
    LoginURL: [string] The address used to log into a website. Usage similar to "EngineUrl". Can contain the placeholders _agentUser_ and _agent_Password_.
  • Icon
    NextLinkStart: [string] Defines the start of the range for the 'next results page' link.
  • Icon
    NextLinkEnd: [string] Defines the end of the range for the 'next results page' link.
  • Icon
    NextLinkName: [string] Defines the name of the link for the 'next results page' (instead of having to use NextLinkStart/NextLinkEnd).
  • Icon
    NoTopics: [array or string] Array of strings or string containing separated words to ignore in the topics list. Add site specific words here.
  • Icon
    OffsetPerPage: [integer] Increase for _agentOffset_ after every page (default: 1).
  • Icon
    Operators: [integer] Bitmap describing the possibilities of the search engine and the required syntax (see below). The default value is 59 (1+2+8+16+32), meaning that AND, OR, NOT, phrases, and parentheses are supported.
  • Icon
    OperatorsDictionary: [dict] A dictionary containing key-string pairs to convert the name of operators to a different name required by the search engine. Possible keys are AND, ANDNOT, OR, NOT, XOR, NEAR, BEFORE, AFTER, -.
  • Icon
    ParseLinks: [Boolean] Parse and unify links of common search engines (Google, Yahoo, Bing) (default: false).
  • Icon
    Post: [dict or string] Send a HTTP POST instead of a HTTP GET action using this form data. Note: The strings of the form data can contain _agentQuery_ and _agentNumber_ placeholders.
  • Icon
    PostScan: [Boolean] Tell DEVONagent Pro to check if the retrieved pages match the search term (true), or to accept all (false) pages (default: true).
  • Icon
    ResultsPerPage: [integer] Number of results per page and maximal value for _agentNumber_ (default: 10).
  • Icon
    RequiresCrawling: [Boolean] Marks this plugin as needing crawling.
  • Icon
    RequiresThumbnails: [Boolean] Marks this plugin as using only thumbnailed links. Crawling needs to be skipped.
  • Icon
    RoundResultsPerPage: [integer] Always round number of results per page (_agentNumber_) to multiples of this value (default: 10).
  • Icon
    ScanTerm: [string] String specifying a term used to filter results (e.g. instead of or in addition to the secondary query defined by the user).
  • Icon
    SkipsCrawling: [Boolean] Marks this plugin as skipping crawling.
  • Icon
    Start: [integer] The initial value for _agentOffset_ (default: 0).
  • Icon
    StripTags: [array] Array of strings marking HTML tags defining blocks to strip.
  • Icon
    TextEnd: [string or array of strings] String marking the end of the range of the HTML source code containing the relevant text of a page. TextStart is optional if TextEnd is specified.
  • Icon
    TextStart: [string or array of strings] String marking the beginning of the range of the HTML source code containing the relevant text of a page. Note that TextEnd is optional if TextStart is specified. Useful if you're querying a specific site (all results have the same layout) to skip headers, footers or navigation of a page. This improves plain text display, digest and summarization.
  • Icon
    TextTags: [array] Array of strings marking HTML tags defining blocks to convert to text.
  • Icon
    Timeout: [integer] Sets the timeout for requests; default is 60 seconds.
  • Icon
    TitleEnd: [string or array of strings] String marking the end of the range of the HTML source code containing the title of a page.
  • Icon
    TitleStart: [string or array of strings] String marking the beginning of the range of the HTML source code containing the title of a page. Note that TitleEnd is optional if TitleStart is specified. Only useful if all results have identical or no titles (for example a Usenet query).
  • Icon
    UseListingResults: [Boolean] Treat the returned pages as results (and not as pages linking to the results). All Links... keys are unused in that case.
  • Icon
    UserAgent: [string] Custom HTTP user agent string.

Keys for XML/JSON

Required Keys:

  • Icon
    ResultsKeyPath: [string] The path to the results in the search engine's output.
  • Icon
    LinksKeyPath: [string] The path to the links relative to ResultsKeyPath.

Optional Keys:

  • Icon
    ContentsKeyPath: [string] The path to the contents relative to ResultsKeyPath.
  • Icon
    DescriptionsKeyPath: [string] The path to the descriptions relative to ResultsKeyPath.
  • Icon
    TitlesKeyPath: [string] The path to the titles relative to ResultsKeyPath.
  • Icon
    ThumbnailsKeyPath: [string] The path to the thumbnails relative to ResultsKeyPath.
  • Icon
    JSONPost: [Dictionary] Sends the content of the dictionary as HTTP POST request. The dictionary can contain nested dictionaries or arrays. string-placeholders, e.g., _agentNumber_ or _agentQuery_, are still replaced.

Localization

Three identifier keys of a plugin can be localized: Name, Info and Description. Specify the desired key as a dict (dictionary) containing key and string pairs, one per additional language. The key contains the appropriate language code, e.g., en, de, or fr, and the string is the localized text for the specified language.

Example:

<key>Name</key>
<dict>
<key>en</key>
<string>DEVONthink Server</string>
<key>de</key>
<string>DEVONthink-Server</string>
</dict>

Additionally, the following keys can be localized by adding a suffix for use with language-specific search engines: EngineUrl, EngineNextUrl, KeywordsUrl, and NextLinkName. Language identifiers are: DE, DA, NL, ES, HU, FR, IT, CS, PT, PL, SV, FI, EN, NO, RU, EL, AR, and HE.

Example:

A URL in the key EngineUrl_DE is used to target the German variant of a search engine.

Notes

Operator Bitmap: The following table gives you the necessary values for calculating the operator bitmap for the "Operators" key (see above):

  • Icon
    0 [value: 1]: Supports AND operator
  • Icon
    1 [value: 2]: Supports OR operator
  • Icon
    2 [value: 4]: Supports XOR operator
  • Icon
    3 [value: 8]: Supports NOT operator
  • Icon
    4 [value: 16]: Supports "" phrases
  • Icon
    5 [value: 32]: Supports () parenthesis
  • Icon
    6 [value: 64]: Supports NEAR operator
  • Icon
    7 [value: 128]: Supports AFTER operator
  • Icon
    8 [value: 256]: Supports BEFORE operator
  • Icon
    9 [value: 512]: Supports NEXT operator
  • Icon
    11 [value: 2048]: Use ANDNOT instead of NOT or -
  • Icon
    12 [value: 4096]: Convert search term to lower case
  • Icon
    13 [value: 8192]: Convert search term to upper case
  • Icon
    14 [value: 16384]: Engine scans for any word by default (not for all words) without an explicit operator
  • Icon
    15 [value: 32768]: Use AND/NOT instead of +/-
  • Icon
    16 [value: 65536]: Use | instead of OR
  • Icon
    17 [value: 131072]: Supports OPT/ operator

LinksStart, et al.: LinksStart, LinksEnd, LinksMatching, and LinksNotMatching are three different ways to specify interesting links of a page. Specify none, some, or all of them depending on the search engine you are accessing. The more you specify, the higher the chances are that the plugin will still work after the structure of the HTML code or the links changed. First DEVONagent Pro retrieves links using LinksStart/LinksEnd, then it filters them using LinksNotMatching, and finally it matches them against LinksMatching.

Link Listings vs Matched Pages: LinksStart/LinksEnd, LinksMatching, and LinksNotMatching are used on pages listing links, e.g., a page of search engine results. Links are always followed on these pages. TextStart/TextEnd, TitleStart/TitleEnd, and DateStart/DateEnd are used on individual web pages, i.e., a matched Wikipedia page. The FollowLinks key/option can be used on these pages.